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Abstract 

We developed automatic techniques that allow the extraction of physically 
significant parameters from auroral images. This allows the processing o a 
much larger number of images than is currently possible with manual tech- 
niques. We applied our techniques to diverse auroral image datasets. . e 
made these results available to geophysicists at NASA and at universities 
in the form of a software system that performs the image analysis.^ After 
some feedback from users, we transfered an upgraded system to NASA an 
to two universities. We demonstrated the feasibility of user traine searc 
and retrieval of large amounts of image data using our automatically derived 
parameter indeces. We developed and applied techniques based on classifi- 
cation and regression trees (CART) to broaden the types of images to which 
the automated search and retrieval may be applied. We tested our techniques 
with DE-1 auroral images. 
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1 Research results for the project 
1.1 Introduction 

Our CESDIS research involved a collaboration between computer scientists 
and electrical engineers interested in researching and understanding practica 
computer image analysis algorithms and geophysicists interested in the use 
of these tools to enable them to study the vast quantities of data available 
from NASA satellites. The project provided benefits to both communities 
of researchers. It allowed the image analysis researchers to make realistic 
assumptions about the data during their research on the algorithms The 
ability to test the algorithms with large sets of satellite images ensured that 
the “mht : ’ problems were solved. The geophysicists in turn gained tools t at 
allows them to quickly study the vast amounts of data that would otherwise 
remain unused. This final report describes the results of the past several 
years of research in developing and evaluating the tools to process images of 
the Earth's aurora, and in transfering these tools to the users. 

1.2 Developing the tools 

The fir<t task of the project was to obtain the data: DE-1 satellite images 
from the University of Iowa and VIKING satellite images from the University 
of Calgary. The necessary infrastructure was built m order to manipulate 
these images with the Stanford image processing systems. The decision was 
made to concentrate first on one data set to demonstrate the feasibility of 
computer processing of these types of auroral images. The DE-1 satellite 
images were selected because over 500,000 have been gathered. We obtained 
1000 of these images for our tests and we gained the experience necessary to 

process them. 

1.2.1 Standard techniques 

Many of the standard techniques available from image processing and com- 
puter vision were applied to the DE-1 images. These standard techniques did 
not provide for adequate performance with these images. The problems that 
were identified were the uneven solar illumination in the images, the dynamic 
changes in shape and location of the aurora and the requirement for many 
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diverse measurements and parameters by the geophysicists [Ref. 1,4]. It was 
determined that recovering the inner auroral boundary and the outer auro- 
ral boundary would provide a description of the images rich enough to allow 
many different measurements and parameters to be calculated. We began 
working on finding the inner auroral boundary using the computer. This in- 
ner boundary allows the determination of the magnetic field within the polar 
cap (the region within the inner boundary of the aurora). This magnetic fiel 
quantity is a fundamental parameter needed by geophysicists to study the 
flow of energy through the solar wind - magnetosphere - ionosphere system. 

1.2.2 Newly developed techniques for boundary finding 

Scientific images require a different set of assumptions than those for other 
computer vision applications. The assumptions of rigid bodies and of simple 
shapes for objects is valid for many industrial applications, but it is not 
valid for the auroral images (nor for most other scientific applications). For 
this reason, the decision was made to extend and apply to the images a 
technique called “snakes*, the fitting of an elastic curve to the data, for 
finding the boundaries. Several problems arose during the application of 
makes to the auroral images. Our contributions to the solutions of these 
problems include the extension of snakes to apply to objects with changing 
topologies, the understanding of the stability of snakes, and the development 
of “adaptive snakes" that allow the automatic setting of snake parameters. 
These extensions substantially increase the number of applications for vhici 
snakes are a viable computer processing technique [Refs. 6,7]. Recently, a 
medical imaging company has adapted the algorithm described in [Ref. 6] 
for possible use with their ultrasound instruments. 

The snakes were tested extensively with the DE images. The results were 
compared to manual measurements, which were used as a “gold' 5 standar 
[Refs. 8,9,10]. The performance of the snakes was found to be adequate for 
winter hemisphere scenes, but it was not adequate for the summer hemisphere 
scenes. 

For this reason, a new optimization approach to the boundary finding 
was developed based on a cartesian to polar conversion of the data. The new 
technique first finds the best fit ellipse, in a weighted least square sense, to 
the image data. This best fit ellipse is then used as an initial estimate for a 
second stage of processing that removes the requirement that the fit be to an 
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ellipse. The second stage has been developed in two alternate ways, the first 
using the snakes previously described, and the second using a line-by-line 
processing technique in the polar coordinate domain. This new technique 
allows the extraction of the auroral boundaries for both winter and summer 
scenes. We show an example of the application of this technique in the next 

section. 

We also developed interactive methods for finding boundaries. We wrote 
an X-windows based software system for interactive modification of bound- 
aries. This was used both to generate manual boundaries for use as gold 
standards, and also for modification of automatically generated boundaries, 
when that was needed. Finally, we developed a novel way to find boundaries 
from images interactively by combining standard image processing techniques 
with manual techniques [Ref. 3]. 

1.2.3 Newly developed image retrieval techniques 

As mentioned above, the automation of the boundary extraction step in au- 
roral image analysis allows the extraction of quantitative parameters from 
much larger sets of images. The user naturally wishes to query these large 
databases on the basis of the extracted quantitative parameters. Since the 
users’ requirements vary from person to person, and from day to day, we 
developed algorithms and techniques that allow the user-trained query of in- 
formation based on image content [Ref. 4], Our work here is complementary 
to image browsing systems that are currently under development by others. 
Our work offers the capability of search and retrieval based on quantitative 
measures from the images. This extends the capabilities of most browse 
systems that do not allow quantitative extraction of information. 

We developed algorithms based on classification and regression trees (CART) 
[Ref. 4] to select images from the larger database. CART combines many 
unique abilities: it can handle hundreds of variables, including both non- 
numerical and numerical ones, processes missing data in a simple, effective 
manner, it is designed to perform well with previously unseen data, it is ef- 
ficient, it is trained by examples and it supports the users’ understanding of 
the data selection decisions. This last point is most important: CART can 
be used in the same manner as neural networks but the results of the CART 
decisions are easily interpreted by the scientific users. 

We used CART to combine many diverse parameters and features that 
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were derived by our auroral boundary extraction system. We used the CART 
algorithm, together with required auroral parameter extraction interfaces re- 
quired to drive the CART system. An interactive parameter selection system 
based on X- windows was written. This facilitated the formation of queries 
by the user. The user “queries” the system by providing examples of images 
or parameters of interest. The system “replies” by retrieving datasets similar 
to the ones presented during the query. The performance of the system was 
tested by using gold standards provided by manual replies extracted for the 
queries [Ref 4b We used many DE satellite images for this phase of the study. 


2 Applying the tools 

The measurement of the inner boundary can be used to derive the magnetic 
flux within the polar cap. This quantity is a fundamental parameter neede 
bv the geophvsicists. There is a need to convert the results to the proper 
geometries and units of measurement. Two summer students helped with 
these tasks. Steve Ross developed a system to remap the auroral image 
sequences to image sequences from a fixed viewpoint. This system allows the 
motion due to the satellite to be removed, leaving only the residua motion 
due to the dynamics of the aurora. Chacko Sonny started developing a 
system to allow the measurements of surface area and the incorporation o a 
magnetic field model to allow the determination of magnetic flux. Domingo 
Mihovilovic completed this work to allow the extraction of magnetic flux m 
Webers. This, together with an estimate of the area of the polar cap, al ows 
the recovery of information useful to the scientists, «.g. the magnetic flux 
through the polar cap region. 

2.1 Technology transfer of auroral image analysis 

With the help of Dr. Robert McGuire of NSSDC at Goddard we identified the 
Coordinated Data Analysis Workshop (CDAW) 9 dataset as a useful dataset 
for testing our techniques and for providing our results to the NASA scientific 
community. We obtained the CDAW 9 dataset from Dr. John Craven of 
the University of Iowa. We completed of our analyses with this dataset 
and provided the processed information to Dr. Clauer at the University o 
Michihgan, Dr. McGuire at NSSDC and to Dr. Craven at the University of 
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Iowa for their evaluation. 

Figure 1 shows part of the CDAW 9 dataset to which we have applied our 
polar coordinates algorithm. This figure shows a sequence of fifteen DE-1 
images gathered on May 4, 1986. The sequence is displayed in column scan 
order, from top left to bottom left and then from top right to bottom right. 
Overlayed on the images are the inner boundaries automatically generated 
u-bng our polar coordinates processing system. The inner boundaries are 
adequately found in all of these images. The boundaries appear to grow as 
time progresses but this is partly due to the satellite moving closer to the 
Earth Removing the effects of the satellite motion and using the IGR 
magnetic field model for the Earth allows us to provide physical parameters 
from our automated processing. Figure 2 shows a time series plot of the 
integrated magnetic flux within the auroral oval for each of the fifteen images. 
Theremoval of the satellite motion reveals the underlying temporal changes 

in the magnetic field strengths. veenr 

Dr. McGuire of NSSDC aided us in installing our software at AbbDL, onto 

their sun workstation. Dr. Robert Clauer made contact with Dr. Marsha 
Torr of NASA Marshall Space Flight Center regarding the application o 
our techniques to the auroral images expected from the upcoming Glo a 
Geospace Sciences (GGS) mission. Our work was discussed at two of their 
ultraviolet imaging team meetings. We had interest expressed by Dr. Richard 
Savage of Hughes in Colorado regarding applications of our techniques o 
images obtained by the DMSP satellite. 

2.2 Other technology transfer and collaboration 

Throughout this CESDIS project we benefited from the many contacts made 
at Goddard. For Dr. Ed Kemper we developed a new algorithm to enhance 
noisy X-ray images of diffuse X-ray scattered. With Dr. Josephmo Comiso 
we have had interactions that have resulted in a joint publication of applica- 
tion of statistical analyses comparing Synthetic Aperture Radar images wit 

laser altimeter data. , , , ~ n 

Among others that we have had interactions with at Goddard are Dr. Dan 

Baker, Laboratory for Extraterrestrial Physics, Drs. Robert Price, Jim Tilton 
of Science Information Systems, Dr. David Leisawitz regarding astronomical 
applications, Drs. Jim Green, Bob McGuire, Owen Storey, Lloyd Tremish 
and Robert Cromp of NSSDC. 
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Figure 1 

Part of the CDAW 9 dataset showing a sequence of fifteen DE-1 satellite 
images of the aurora. The results of automatic inner boundary 
estimation are overlayed. 
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Figure 2 

Time series plot of the integrated magnetic field intensity for the fifteen 

images in Figure 1 . 




CESDIS facilitated helpful interactions with Drs. Aznel Rosenfeld Peter 
Meet and Behzad Kamgar-Parsi then of the University of Maryland. \\e «ere 
also happy to invite several people to CESDIS workshops. Dr. Peter Burt 
of SRI Sarnoff laboratory, Dr. Patrick Newell and Mr. Simon Wing of the 
Applied Physics Laboratory, Dr. Ramesh Jam of the University of Michiga 
and Dr. Glen Langdon of the University of California at Santa Cruz. 

2.3 Technology transfer via educational activities 

Educating the next generation of researchers was an important part of our 
technology transfer activities. Student Mihovilovic finished his PhD funded 
partially by CESDIS. Masters students Cecilia Han and Lalit Katragadda a 
cummer students Steve Ross and Chacko Sonny contributed to the project 
with CESDIS funding. Several people volunteered their time for this projec 
without any CESDIS funding. We wish to acknowledge them here Jason 
Daida (has completed PhD in Electrical Engineering) who applied iterati 
segmentation techniques to the auroral images m order to > im P ro ' e . th «^"|' 
mertime scenes [Ref. 14], Raymond Wong (PhD candidate m Electrical Engi- 
neering) worked on a pattern recognition approach to image retne^ al an . r. 
Heping Zhang (has completed PhD in Statistics) who aided us briefly during 
theearly stages of the aurora application in the use of Markov Random Fiel 

estimation. . , + + i tt^; 

Dr. Samadani taught twice a graduate image processing class at the Uni- 
versity of California, Santa Cruz, and once at Stanford University ese 
three courses included examples of applications to several NASA remote sens- 
ing problems, including our CESDIS auroral image analysis work. 


3 Conclusions 

We made considerable progress in removing the bottleneck created by the 
manual extraction of boundaries from satellite images of the aurora. Our sys- 
tem allows the extraction of physically significant parameters from a much 
larger number of images than is feasible manually. Much larger scien i 
studies may now be conducted using our automated methods. The ' organ) 
tion, search and retrieval of the images needed for these laager stud.es m turn 
became the new challenge that we addressed towards the end of our CES- 
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DIS research We provided the software for image analysis to four different 
locations: the University of Michigan, under supervision of Dr Clauer, the 
University of Iowa, under supervision of Dr. Frank, to the CESDIS compu er 
center and to the NSSDC computer center. The last two centers are locate 
at Goddard Space Flight Center. 


4 Software 

This section describes briefly the programs used to analyze the auroral im- 
ages. For additional information about the software, contact the CLbUlb 
office at Goddard. 

In the main directory of the tree where the software resides, there are 
two files that new users should read. The first is the README e t at 
contains an annotated listing of the tree and the second is the IN Si ALL h e 
that describes installation of the software. The f?7 and gnu C compiler are 
needed for compilation, as well as XI 1 software. A user familiar with Unix, 
X-Windows, C and C-shell scripts should aid the users during the installation 

process. 

xwframe This program reads an image file in hips format from standard 
input (stdin) and displays it on the screen of a system running X Win- 

dows. 

Usage : xwframe < filename 
options : 

-n will display the given string on the window top bar. It defaults to 
the name of the hips image (read from the header of the hips file). 

-x defines the x location of the xwframe window. 

-y defines the y location of the xwframe window. 

xcolor This program allows the user to change the X Windows colormap 
that is shared by xwframe. Xcolor must be started before xwframe is 

called. 

Usage : xcolor 
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maftohips This program reads a file in maf format from standard input 
(stdin) and writes the corresponding image in hips format to standard 

output (stdout). 

Usage : maftohips < filename. maf > filename.hpl 

correct This program converts a file from maf format to a format that is 
compatible with the way that data records are read in Fortran {cmaj 
format). The program appends at the beginning and at the end of each 
original record four bytes that contain the record length. 

Usage : correct < filename. maf > filename. cmaf 

read This program reads a file in cmaf format and saves the header infor- 
mation and (latitude, longitude) pairs for each pixel m the image as 
ASCII text to a file with extension PIX. It is hardwired to read from 
“tempor.maP (cmaf format file!) and write the result to “temper. PIX' . 

Usage : read 

latlonhips This program reads a PIX file (“filename. PIX'’) and it gener- 
ates two floating point hips files containing the latitude and longitude 
information. The two output files are called “filenamelat.hpl ' and file- 
namelon.hpF. These two images must be converted to byte hips for- 
mat before they can be displayed by xwframe. The conversion and 
display can be done at once with the command scale < filenamelat.hpl 
| xwframe for the latitude image and similarly for the longitude image 
: scale < filenamelon.hpl | xwframe 
Usage : latlonhips filename 

readheader This program reads a maf file from standard input (stdin) and 
writes the header information in ASCII to standard output (stdout). 

Usage : readheader < filename. maf > filename.hdr 

polar This program converts an image in hips format to its polar representa- 
tion. The center for the Cartesian to polar coordinates transformation 
is read from the file “filename'xOrCF that contains the x and y coordi- 
nates of the center in ASCII. The option -n allows to define the number 
of rows of the transformed image. The option -r allows to define the 
number of columns of the transformed image. 
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Usage: polar < filename. hpl > filename. p 
options : 


-n defines the number of rows of the output image. 

-r defines the number of columns of the output image. 


editor This program is used to edit/generate interactively auroral bound- 
aries. It reads automatically the hips image (“filename.hP 1 ' ), and the 
inner and outer boundaries ( “filename.insnk” and "filename.outsn • 
respectively). It also uses the latitude and longitude information ( hle- 
namelat.hpP and •filenamelon.hpl’’ files respectively) 


Usage: edit -f filename 
options : 


-f defines the filename 

-cellsize defines the magnification factor for the image 

find This program searches for the auroral boundaries. It reads the po- 
lar image from standard input (stdin). It also uses the latitude and 
longitude maps (“filenamelat.h P r and “filenamelon.hpl files respec- 
tively) The boundaries are saved to the files “filename. msn • an 
“filename.outsnkT The program real-area can then be used to com- 
pute the area and total integrated magnetic field inside any of th 
boundaries. 

Usage: find < filename. p > /dev/null 
options : 

-x use X graphical interface. By default, it runs on the background. 

real -area This program computes the area and the total magnetic field 
inside boundary. The program reads the boundary file from ffle 
name. mb’ and it uses the latitude and longitude images ( 
lat.hpl” and “filenamelon.hpl”). The files with the coefficients for the 
magnetic field model most be in the current directory These fi e 
are dgrf65.dat, dgrf75.dat, igrf45.dat, igrf55.dat, igrfS5.dat, dgrfiO.da , 
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dgrfSO.dat, igrf50.dat, igrf60.dat and igrfS5s.dat. The total number of 
pixels, total area in km 2 and the total integrated magnetic field are 
written to stdio and to a file called “filename.mag'’. 

Usage : real_area filename > /dev/null 

run -file.csh This shell file generates the hpl, lat.hpl , lon.hpl, and hdr files 
from a given maf file. The lat.hpl and lon.hpl files are saved m com- 
pressed format (using the standard compress Unix command). 

Usage : run-file. csh filename 

gen-polar. csh This shell is used to generate the polar plot and the cOrO files 
manually. The hpl file is displayed, then the user clicks m the center of 
the aurora and types in the center estimate coordinates (cO.rO). Using 
this location as the estimate of the aurora center the polar image is 
created with extension p 
Usage : gen.polar.csh filename 


Glossary of file formats 
maf DESAI image format. 

cmaf DESAI image format corrected for proper handling by Fortran pro- 
grams. 

hpl HIPS image format. HIPS stands for Human Information Processing 
Laboratory's Image Processing System. 

PIX File containing the header information plus (latitude.longitude) infor- 
mation for each pixel in ASCII format. 

hdr File containing all the header information in ASCII format. 

cOrO File containing x and y coordinates of the estimate of the center m 

ASCII. 
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5 Personnel 

The personnel that worked on this research were Professor G. Wiederhold as 
principal investigator from 19S9 - 1991. Professor Wiederhold was mvohed 
i„ a U idin^ the research and also in aid to CESDIS including proposal review 
and workshop participation. Professor M. Flynn was principal investigator 
during 1991 - 1993 and was involved in the management of this project. Dr. 
Samadani worked on the research throughout the funding perio . e was 
responsible for overall supervision of the development of the techniques for 
auroral image analysis. Dr. Robert Clauer guided the geophysical aspects o 
the work while at Stanford. Dr. Clauer currently works at the University 
of Michigan. From Michigan, he continued to provide geophysical guidance 
through visits to Stanford and via electronic network collaboration. As noted 
above! several graduate students contributed to this research. 


6 Publications and presentations 

A list of publications from the CESDIS research is included below. Most of 
these reports are available from CESDIS as technical reports. 

Our CESDIS work was advertised in three places: the cover of the pro- 
aram <mide for SPlE's Februrary 1991 Image Processing conference in Santa 
Clara°shows a photograph from our CESDIS research, the bSRA 20th an- 
niversary publication included a reprint of the IEEE journal pubbcation of 
our CESDIS funded research and we were interviewed about our LLbDIP 
research for an article that appeared in the USRA Quarterly. 

Publications and Papers Presented at Professional Meet- 
ings resulting from the CESDIS research 

[11 G. Wiederhold, J. Brinkley, R. Samadani and C.R. Clauer, “Model-Driven 
Image Analysis to Augment Databases’’, Invited paper in T. Kunn (editor), 

Springer Verlag, Tokyo, 19S9, pp. 159-180. . .. . ,, . 

[2] R. Samadani, “A Tutorial on Image Processing and Computer V ision , m 
Advanced Computer Methods for Systematic Biology, John Hopkins press, 

1993. 
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[3] R. Samadani and C. Han, “Computer Assisted Extraction of B° undar ‘ es 

SK KSl ^ont en^ed E^tron^ 

1993 San Jose. California, Vol. 190b, pp. 50 - 59. _ , f 

[51 D. Mihovilovicand R. Samadani. -‘A Minimization-Pruning Algorithm ic 
Finding elliptical boundaries in images with non-constant ac 'o rou 
with missing data". SPIE Image Processing Algorithms and Techniques II , 
February 1992. San Jose. CA, Vol. 1657, pp. 546-557. 

[6] R. Samadani, "Adaptive snakes: control of damping and material param- 
eters". SPIE conference, Geometric Models in Computer ision, u . 

San Die^o. CA. Vol. 1570. pp. 202-213 

[7] R. Samadani. "Changes in Connectivity m Active Contour . ° e s ro 

ceedings of the IEEE Workshop on Visual Motion, March I9b9, ’ * •’ 

[£) R 33 Samadani. D. Mihovilovic. C.R. Clauer, G. Wiederhold, XD. Craven 
and L A Frank. “Evaluation of an Elastic Curve Technique for Finding the 
Auroral Oval from Satellite Images" MEEE Transactions on Geoscience and 

Remote Sensing. Julv 1990. pp- 590-597. . . 

[9] D Mihovilovic A.'. R. Samadani and C.R. Clauer, “Evaluation of an Elas- 
tic Curve Technique for Finding the Auroral Oval from Satellite Images 
Proceedings of the International Geoscience and Remote Sensing Sympo- 
sium, July 19S9. Vancouver, Canada, pp,126S- 12/2^ Traven 

[101 R. Samadani, D. Mihovilovic. C.R. Clauer, G Wiederhold, J_D. Cra en 
and L.A. Frank, “A Computer Vision System for Automatica y m g 
Auroral Oval from Satellite Images", SPIE/SPSE Conference on Image Pro- 
cessing Algorithms and Techniques. Feb.12-14, 1990, Santa Clara, California, 

m] M.c! Walker, C.R. Clauer, R. Samadani, J.D. Craven and L.A Frank 
“Automated Analysis of Auroral Images’ , American Geophysical Union 
Conference, 1987, San Francisco, California. See EOS Transactions, Vol. 
gg j^o. 44, November 3, 1987, pg. 1436 for abstract- , r 

[19] C.R. Clauer, R. Samadani, D. Mihovilovic, J. \esec -y, • an - , • 

Wiederhold, J. Craven and L. Frank, “Computer assisted analysis of auroral 
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images obtained from high altitude polar satellites” Presented at the sixth 
scientific assembly of 1AGA. Exeter, U.K, August, 1 • R , G 

[13] C.R. Clauer, R. Samadani, D. Mihovilovic, J. Vesecky, • ’ 

Wiederhold, J. Craven and L. Frank, "Conner automated of au- 
roral images obtained from high altitude polar satellites , EOS, Trans. A ^ 

Geophvs. Union, 70, 405, April, 19S9 (abstract). Presented at the Spn e 

Meeting of the American Geophysical Anion, May, 19S9. 

114] J. Daida, R. Samadani and J.F, Vesecky, "Adaptive Image i Segmental, on 
Applied to Extracting the Auroral Oval from Satellite Images , Proceedings 
of the International Geoscience ar.d Remote Sensing Symposium, May -0 ■ 
24, 1990, Washington D.C., pp. 1649 - 1650. 

Other Presentations of our CESDIS research results 

Presentations of the CESDIS research have been made at many place*, in- 
eluding NASA Goddard at various CESDIS seminars, and the Goddard Ad- 
vances^ Computer Science seminar, University of Iowa Physics de P artI ” en ’ 
University of Michigan Electrical Engineering department, Unnerstty of Ca 
fornia Santa Cruz. Computer Engineering department, IEEE Geoscience and 
Remote Sensing Society of Santa Clara. California, Stanford University sem- 
inars in Electrical Engineering, Geophysics and Earth - fences an ^ an i . 
Workshop on advanced computer methods for biology in Napa Californi . 
The other presentations at conferences are shown m the publications E. 

above. 
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